The paradigm of discrete dynamic programming in stochastic investment and production problems

Orientador: João Bosco Ribeiro do ValTese (doutorado) - Universidade Estadual de Campinas, Faculdade de Engenharia Eletrica e de ComputaçãoResumo: Apresenta-se um modelo de controle por intervenções para o problema de produção e estoque de vários itens, com diversos estágios de produção. Este problema pode ser solucionado via programação dinâmica discreta (PD) por um operador de custo descontado. Para contornar a dificuldade de obtenção da solução ótima via PD ao se considerar um número razoável de classes de itens e suas etapas de produção, esta tese desenvolve-se em duas linhas. A primeira delas consiste em tomar uma noção de estabilidade estocástica no sentido Foster-Lyapunov para caracterizar a família de soluções candidatas a ótima, originando uma classe de políticas que geram um subconjunto de estados que são recorrentes positivos. Dessa forma, é possível propor políticas sub-ótimas que sejam estáveis, e cuja consideração de otimalidade possa ser desenvolvida apenas no subconjunto de estados recorrentes, simplificando a tarefa da PD e focando nos estados mais freqüentados no longo prazo. A segunda linha de abordagem consiste em desenvolver técnicas de PD aproximada para o problema, através de uma arquitetura de aproximação fixa aplicada a um subconjunto amostra do espaço de estados. Um avanço analítico é alcançado por observar como uma arquitetura de aproximação pode capturar adequadamente a função valor do problema, vista como uma projeção da função valor na arquitetura. Condições para que um algoritmo de PD aproximada convirja para essa projeção são obtidas. Essas condições são independentes da arquitetura utilizada. Um algoritmo derivado dessa análise é proposto, a partir do monitoramento da variação de passos sucessivosAbstract: We propose an intervention control model for a multi-product, multi-stage, single machine production and storage problem. The optimal policy is obtained by means of discrete dynamic programming (DP), through a discounted cost contraction mapping. In order to overcome the difficulty of obtaining the optimal solution for problems with a reasonable number of products and production stages, we take two different approaches. The first one consists in using a notion of stochastic stability in the Foster-Lyapunov sense to characterize the candidate policies, thus originating a class of policies that induce a subset of positive recurrent states. Therefore, one can propose suboptimal policies that are stable and seek optimality only in the subset of recurrent states, in such a way that simplifies the DP task and focuses on the states which are visited more frequently in the long run. The second approach consists in developing approximate dynamic programming techniques for the problem, by means of a fixed approximation architecture applied to a sample subset of the state space. A novel result is obtained by observing how an approximation architecture can adequately capture the value function of the problem, which is viewed as a projection of the value function into the architecture. We obtain conditions for an approximate DP algorithm to converge to this projection. These conditions are architecture independent. An algorithm derived from this analysis is proposed that monitors the variation between successive iteratesDoutoradoAutomação e ControleDoutor em Engenharia Elétric

A multi-cluster time aggregation approach for Markov chains

This work focuses on the computation of the steady state distribution of a Markov chain, making use of an embedding algorithm. In this regard, a well-known approach dubbed time aggregation has been proposed in Cao et al., (2002). Roughly, the idea hinges on the partition of the state space into two subsets. The linchpin in this partitioning process is a small subset of states, selected to be the state space of the aggregated process, which will account for the state space of the embedded semi-Markov process. Although this approach has provided an interesting body of theoretical results and advanced in the study of the so-called curse of dimensionality, one is still left with a high-dimensional problem to be solved. In this paper we investigate the possibility to remedy this problem by proposing a time aggregation approach with multiple subsets. This is achieved by devising a decomposition algorithm which makes use of a partition scheme to evaluate the steady state probabilities of the chain. Besides the convergence proof of the algorithm, we prove also a result for the cardinality of the partition, vis-à-vis the computational effort of the algorithm, for the case in which the state space is partitioned in a collection of subsets of the same cardinality

Dimensionality reduction for multi-criteria problems: an application to the decommissioning of oil and gas installations

This paper is motivated by decommissioning studies in the field of oil and gas, which comprise a very large number of installations and are of interest to a large number of stakeholders. Generally, the problem gives rise to complicated multi-criteria decision aid tools that rely upon the costly evaluation of multiple criteria for every piece of equipment. We propose the use of machine learning techniques to reduce the number of criteria by feature selection, thereby reducing the number of required evaluations and producing a simplified decision aid tool with no sacrifice in performance. In addition, we also propose the use of machine learning to explore the patterns of the multi-criteria decision aid tool in a training set. Hence, we predict the outcome of the analysis for the remaining pieces of equipment, effectively replacing the multi-criteria analysis by the computational intelligence acquired from running it in the training set. Computational experiments illustrate the effectiveness of the proposed approach

Optimal testing policies for diagnosing patients with intermediary probability of disease

This paper proposes a stochastic shortest path approach to find an optimal sequence of tests to confirm or discard a disease, for any prescribed optimality criterion. The idea is to select the best sequence in which to apply a series of available tests, with a view at reaching a diagnosis with minimum expenditure of resources. The proposed approach derives an optimal policy whereby the decision maker is provided with a test strategy for each a priori probability of disease, aiming to reach posterior probabilities that warrant either immediate treatment or a not-ill diagnosis

Optimisation and control of the supply of blood bags in hemotherapic centres via Markov Decision Process with discounted arrival rate

Running a cost-effective human blood transfusion supply chain challenges decision makers in blood services world-wide. In this paper, we develop a Markov decision process with the objective of minimising the overall costs of internal and external collections, storing, producing and disposing of blood bags, whilst explicitly considering the probability that a donated blog bag will perish before demanded. The model finds an optimal policy to collect additional bags based on the number of bags in stock rather than using information about the age of the oldest item. Using data from the literature, we validate our model and carry out a case study based on data from a large blood supplier in South America. The study helped achieve an overall increase of 4.5% in blood donations in one year

Modeling the integrated mine-to-client supply chain: a survey

Mining is an important economic activity and a highly complex industry. As such, it demands a complex supply chain to connect mines to clients, often involving railways, ports and long-distance maritime shipping. State-of-the-art optimization tools are an invaluable asset to help manage such a complex environment, which makes mining industry a very fertile ground for operational research applications. This paper aims to present a bibliographical review of published works involving operational research applications in the mining industry. We start by mapping applications within each isolated link of the chain. Then, we make inroads into the researches involving and connecting multiple links of the mining chain. Finally, we present summaries of our finding and pinpoint some directions for research opportunities in the mining industry

Difficulties in access and estimates of public beds in intensive care units in the state of Rio de Janeiro

OBJECTIVE To estimate the required number of public beds for adults in intensive care units in the state of Rio de Janeiro to meet the existing demand and compare results with recommendations by the Brazilian Ministry of Health. METHODS The study uses a hybrid model combining time series and queuing theory to predict the demand and estimate the number of required beds. Four patient flow scenarios were considered according to bed requests, percentage of abandonments and average length of stay in intensive care unit beds. The results were plotted against Ministry of Health parameters. Data were obtained from the State Regulation Center from 2010 to 2011. RESULTS There were 33,101 medical requests for 268 regulated intensive care unit beds in Rio de Janeiro. With an average length of stay in regulated ICUs of 11.3 days, there would be a need for 595 active beds to ensure system stability and 628 beds to ensure a maximum waiting time of six hours. Deducting current abandonment rates due to clinical improvement (25.8%), these figures fall to 441 and 417. With an average length of stay of 6.5 days, the number of required beds would be 342 and 366, respectively; deducting abandonment rates, 254 and 275. The Brazilian Ministry of Health establishes a parameter of 118 to 353 beds. Although the number of regulated beds is within the recommended range, an increase in beds of 122.0% is required to guarantee system stability and of 134.0% for a maximum waiting time of six hours. CONCLUSIONS Adequate bed estimation must consider reasons for limited timely access and patient flow management in a scenario that associates prioritization of requests with the lowest average length of stay.OBJETIVO Determinar o número necessário de leitos públicos de unidades de terapia intensiva para adultos no estado do Rio de Janeiro para atender à demanda existente, e comparar os resultados com a recomendação do Ministério da Saúde. MÉTODOS Seguiu-se modelo híbrido que agrega séries temporais e teoria de filas para prever a demanda e estimar o número de leitos necessários. Foram considerados quatro cenários de fluxo de pacientes, de acordo com as solicitações de vagas, proporção de desistências e tempo médio de permanência no leito de unidade de terapia intensiva. Os resultados foram confrontados com os parâmetros do Ministério da Saúde. Os dados foram obtidos da Central Estadual de Regulação, de 2010 a 2011. RESULTADOS Houve 33.101 solicitações médicas para 268 leitos de unidade de terapia intensiva regulados no Rio de Janeiro. Com tempo médio de permanência das unidades de terapia intensiva reguladas de 11,3 dias, haveria necessidade de 595 leitos ativos para garantir a estabilidade do sistema e 628 leitos para o tempo máximo na fila de seis horas. Deduzidas as atuais taxas de desistência por melhora clínica (25,8%), estes números caem para 441 e 471. Com tempo médio de permanência de 6,5 dias, o número necessário seria de 342 e 366 leitos, respectivamente; deduzidas as taxas de desistência, de 254 e 275. O Ministério da Saúde estabelece parâmetro de 118 a 353 leitos. Embora o número de leitos regulados esteja na faixa recomendada, necessita-se incremento de 122,0% de leitos para garantir a estabilidade do sistema e de 134,0% para um tempo máximo de espera de seis horas. CONCLUSÕES O dimensionamento adequado de leitos deve considerar os motivos de limitações de acesso oportuno e a gestão do fluxo de pacientes em um cenário que associa priorização das solicitações com menor tempo médio de permanência

Optimal approximation schedules for a class of iterative algorithms, with an application to multigrid value iteration

Many iterative algorithms employ operators which are difficult to evaluate exactly, but for which a graduated range of approximations exist. In such cases, coarse-to-fine algorithms are often used, in which a crude initial operator approximation is gradually refined with new iterations. In such algorithms, because the computational complexity increases over iterations, the algorithm's convergence rate is properly calculated with respect to cumulative computation time. This suggests the problem of determining an optimal rate of refinement for the operator approximation. This paper shows that, for linearly convergent algorithm, the optimal rate of refinement approaches the rate of convergence of the exact algorithm itself, regardless of the tolerance-complexity relationship. We illustrate this result with an analysis of coarse-to-fine grid algorithms for Markov decision processes with continuous state spaces. Using the methods proposed here we deduce an algorithm that presents optimal complexity results and consists solely of a non-adaptive schedule for the gradual decrease of grid size.</p

Optimal control model for vaccination against H1N1 flu

This paper introduces a mathematical model to describe the dynamics of the spread of H1N1 flu in a human population. The model is comprised of a system of ordinary differential equations that involve susceptible, exposed, infected and recovered/immune individuals. The distinguishing feature in the proposed model with respect to other models in the literature is that it takes into account the possibility of infection due to immunity loss over time. The acquired immunity comes from self-recovery or via vaccination. Furthermore, the proposed model strives to find an optimal vaccination strategy by means of an optimal control problem and Pontryagin’s Maximum Principle